A strong regularization on a hybrid MLP/RBF architecture achieves small bias and small variance error

We introduce a Forward Backward and model selection algorithm for constructing a hybrid network of radial and perceptron hidden units for regression. The algorithm determines if a radial or a perceptron unit is required at a given region of input space. Given an error target, the algorithm also determines the number of hidden units. Then the algorithm uses model selection criteria and prunes unnecessary weights. This results in a final architecture which is often much smaller than an RBF network or a MLP.